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Chest diseases are among the most common diseases today. More than one 
million people with pneumonia enter the hospital, and about 50,000 people 
die annually in the U.S. alone. Also, Coronavirus disease (COVID-19) is a 
risky disease that threatens the health by affecting the lungs of many people 
around the world. Chest X-ray and CT-scan images are the radiological 
imaging that can be helpful to detect COVID-19. A radiologist would need 
to compare a patient's image with the most similar images. Content-based 
image retrieval in terms of medical images offers such a facility based on 
visual feature descriptor and similarity measurements. In this paper, a 
retrieval algorithm was developed to tackle such challenges based on deep 
convolutional neural networks (e.g., ResNet-50, AlexNet, and GoogleNet) to 
produce an effective feature descriptor. Also, similarity measures such as 
City block and Cosine were employed to compare two images. Chest X-ray 
and CT-scan datasets used to evaluate the proposed algorithms with a 


highest performance applying ResNet -50 (99% COVID-19 (+) and 98% 
COVID-19 (—)) and GoogleNet (84% COVID-19 (+) and 81% COVID-19 (— 
)) for X-ray and CT-scan respectively. The percentage increased about 1-4% 
when voting was used by a k-nearest neighbor classifier. 
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1. INTRODUCTION 

Recently, the detection of COVID-19, or called Coronavirus becomes very significant and crucial. It 
is important to detect the disease in its early stages. For diagnosing this virus, different approaches are 
proposed and used. For example, medical image mechanisms, blood tests (CBCs), and polymerase chain 
reaction (PCR) have been used. The first medical imaging test, which is used as the primary detection of 
Coronavirus, is reverse-transcription polymerase chain reaction (RT-PCR) as agreed by world health 
organization (WHO). Then, the RT-PCR test has used to help the radiologist to make the final and accurate 
detection. RT-PCR test is considered as very high time consumed, and very risky for people with 
COVID-19 [1]. Therefore, to diagnose COVID-19, it has been recommended to be diagnosed by other 
medical imaging approaches such as X-ray and CT-scan images. 

In various clinics, an X-ray (CXR) chest image has been considered as the first step to identify 
Coronavirus disease. CXR as medical imaging gives immediate diagnosing information about a disease. The 
advantages of using X-ray chest image [1], [2]: i) it is available and widely used in emergency and hospitals, 
ii) it is relatively low cost, iii) it has low risk to human health compared with the other medical imaging 
approaches, and iv) it gives an interpretation without needing an expert radiologist. 
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However, the X-ray approach for diagnosing COVID-19 disaese is considered as a very difficult 
process. The reason behind that radiologists must identify the white spots where water and pus are in these 
images. It takes a long time and not an easy process. Also, the probability of making the wrong diagnosis is 
very common. A radiologist or specialist doctor may commit a mistake to diagnose other diseases. The 
radiologist or specialist doctor may diagnose COVID-19 pulmonary tuberculosis while a patient may suffer 
from COVID-19 [1], [2]. 

On the other hand, CT-scan images can be used instead of X-ray in diagnosing diseases. The 
procedure of taking X-ray may yield a high error rate. Also, CT-scanners are not available for many regions 
such as undeveloped countries compared with X-ray imaging mechanism, disabling its use for quick 
diagnoses. Moreover, taking CT-scan images are very expensive for patients. In CT-scanners, a large volume 
of slices is recorded and provided to each patient suffered from the virus of COVID-19. For the radiologist or 
physicians in diagnosing the virus, the process will make high efforts and workloads on them calling the 
CT-images [1]-[3]. 

Deep convolution neural networks (DCNNs) models are a kind of machine learning that are 
effectively used in processing data for many artificial intelligence (AI) applications. It can give accuracies in 
many tasks such as medical image analysis that can reach the human-level [4]-[6]. On the other hand, 
content-based image retrieval (CBIR) attracted researchers who are interested from different fields such as 
biometric and medical due to huge images that are acquired and saved in database. The idea of CBIR is 
retrieving the most similar images to a query image based on image feature (visual content) when an image 
class label is unavailable. However, CBIR still has two main challenges to tackle with, an effectiveness of the 
feature and short coming of similarity measure between features of query and database images. Hence, it is 
very interested to investigate a recent pandemic COVID-19 in terms of CBIR issues. The most related works 
focus on classification. 

This paper has numerous contributions to the literature. Different DCNNs (Resnet-50, AlexNet, and 
GoogleNet) models are investigated using our setting and a recent segmentation method [7] in the pre- 
processing stage. These models are used in the proposed method to classify chest images and exploit them to 
learn the features of the images. Produced features are evaluated in terms of effectiveness using a framework 
of CBIR. In addition, different similarity measurements are investigated. Exhaustive experiments indicated 
that the GoogleNet model performs accurately and efficiently by generating low features in length (2D). The 
model and City-block distance function made a proposed retrieval algorithm. Consequantly, the clinics 
information that is returned by our retrieval model can help the radiologists in helping them in making a 
decision by giving them a reference in determining a query to the patient’s situation. Also, our model is very 
important for the radiologists and doctors to face the threaten of this disease in case of it comes back. The 
rest of the paper is organized as follows: in the next section some of the related works are presented. 
Section 3 introduces the proposed models that are used in our investigation followed by some experimental 
results and discussion in section 5. Finally, conclusions and some related future works are presented in 
section 6. 


2. RELATED WORKS 

Different DCNNs models have been used in diagnosing chest diseases before COVID-19 appeared 
recently [4]-[8]. For example, in Elshennawy and Ibrahim [4] four different models have been applied: 
ResNet 152V2, MobileNet V2 as pre-trained models, a kind of convolutional neural network (CNN) and a 
long short term memory (LSTM) model. These models were built from scratch for detecting and classifying 
pneumonia. A deep learning (DL) model is also used in Chamveha et al. [5] for automatic chest disease 
detection applied on Thai CXR images. Alternately, there have been extensive works on diagnosing 
COVID-19 patients recently during the outbreak of the disease. In most of these studies, DL networks are 
applied to diagnoise the chest diseases. Systems that use DL models consist of several steps like data 
collection, data preparation, feature extraction and classification, and performance evaluation. Images like 
X-rays or CT-scan are used in these models as they provide fast detection of chest diseases, especially 
COVID-19 [4]-[10]. For example, research by Zhong [9] a DCNN is designed and implemented as a 
classification tool of X-ray images. The network has been used to help the screening of the COVID-19. In 
that work, the authors have used VGG-16 as a CNN. They made some modifications to this network, such as 
reducing the number of layers and filters, add a pooling layer after max-pooling, etc. For chest X-ray scans, 
multi-class classification models based on DL models are proposed in Albahli [10]. The model is used to 
evaluate the performance of chest radiographs to diagnose chest diseases such as COVID-19. By using 
adopting generative adversarial network, which is based on a synthetic dataset, the most significant features 
have been solved. Thus, DL system is useful in finding unique features that cannot be found easily in visual 
recognition. 
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There are some researches use CT-scan images to detect Coronavirus pneumonia disease and 
differentiate it from other pneumonia chest diseases. For example, in Chung et al. [11] an AI system has been 
used where a DL framework called fast-track COVID-19 classification network (FCONet) is proposed. Four 
pre-trained DL models are VGG-16, ResNet-50, Inception-v3, or Exception that are applied to develop 
FCONet. In Dansana et al. [12] a CNN approach is applied to chest medical images dataset in binary 
classification of COVID-19 pneumonia. In that work, different models are implemented such as decision tree 
model, VGG-19 and Inception-V2. Then, a feature detection Kernel is applied for creating feature maps of 
image processing and noise reduction. These feature maps are joined together in order to create vectorized 
feature maps which are fed to VGG-19, Inception-V2, and decision tree models to achieve classification 
operations. Finally, in the last layer, the output will be either classified as COVID-19, normal, or pneumonic. 
For COVID-19 disease, also a deep transfer learning model is proposed [13]. The authors have used 
DenseNet 20 CNN model to classify CT-scan chest images as COVID-19 infected patients (+) or not (-). The 
pre-trained model is used as a tool to extract features by using the learned weights on the ImageNet dataset. 
However, the model achieved 1% more compared to VGG-16 and Resnet 152V2 models. 

On the other hand, unlike classification, CBIR does not need training and testing samples. Instead, 
accurate feature and similarity measures are required to retrieve the most similar images. Hence, medical 
CBIR are applied to retrieve the chest images successfully. For example, a DCNN model as a feature 
generator is created for image retrieval applications [14]. The model is applied on a COVID-19 and non 
COVID-19 X-ray images dataset to retrieve relevant images. Pre-trained ResNet-50, VGG-19 and Inception 
V3 are the deep CNN models which are used in that work. Stacked auto-encoders (SAEs) which is a kind of 
DL method is applied [15] in an efficient as a feature extractor phase for CBIR. In the retrieval phase, the 
image's size is reduced into a smaller and compact representation to compare images faster using a cosine 
distance. COVID-19 images dataset is used and the results show high accuracy in distinguishing COVID-19 
from other diseases. However, generated features from VGG-19, ResNet-50, and Inception V3 models are 
4096 and 2048 dimensions respectively. Research by Zhong et al. [16] a kind of DL called deep metric 
learning is proposed with multi similarity and a sampling approach. It is used to learn a DCNN that is 
designed to diagnose and analysis images collected from chest radiographs of potential COVID-19 patients. 
A CBIR model is then used to extract embedding and offer an attention map of the COVID-19 labels where 
the ResNet-50 model is essentially used to extract feature where the 22" layer of the model was as the first 
stage and the 40" as the second stage. Meanwhile, the rest part and two fully connected layers were the 
projection head. Hence, the extracted features projected into 64 D embedding space. Also, in Hwang et al. 
[17] a fully-automated CBIR is developed by integrating DCNN based pattern classification for CT-scan 
images of diffuse interstitial lung disease (DILD). Obtained feature is 384 D in length. Similar CT-scan 
images as query in DILD is retrieved by evaluating the performance of the proposed CBIR. The similarities 
between the retrieved CT-scan images and the query images are calculated using Euclidean distance function. 
For COVID-19 detection model, an integrated decision support system is used [18]. It is a model for 
classification and retrieval system which has a friendly user interface. In the pre-trained stage, different 
feature dimensions were extracted by ResNet-50, VGG-19, and Denesnet 169 (2048, 4096, and 1024). Then 
deep auto-encoder was used to reconstruct mentioned features to lower dimension codes. Then CBIR method 
uses these deep features using a canonical correlation analysis by combining the features in different feature 
combinations. Similarity measurements are computed using different distance functions (e.g., Euclidean, 
correlation, and mean squared error). Modified Xception model is suggested in Kapadia, and Paunwala [19] 
to implement a simpler DL network based CBIR system. This model is compressed to decrease the number 
of layers. It is proposed to develop the features from medical images. The features of the intermediate layers 
are fused to obtain a feature which is 3504D in length so that the accuracy of the xception model is increased. 
To retrieve the more similar images between a query and database images, Euclidean is applied. Using X-ray 
chest images, DCNN can also be implemented to detect Pneumonia infection in the lung as [20]. Modified 
VGG-19 Net is proposed as DCNN to identify the Pneumonia infection. In that work; about 12,000 images 
from eight chest X-ray dataset were used. Also, CBIR method was used to annotate the images by using 
Metadata and other contents. So far, the GoogleNet model was not investigated and analyzed related to 
COVID-19 based on our knowledge as we have done in this paper. The model produced an effective feature 
with low dimension (2D) that was used with the City-block distance function to retrieve required images. 


3. DEEP LEARNING NETWORKS FOR MEDICAL IMAGES 

Medical images analysis by using machine learning is considered as an active field of research since 
the database used is usually structured and labeled. It is significant to process these images using machine 
learning since it improves patient health, and it gives a testbed for an interactivity between the human and AI. 
Examples of digital medical images; that can be processed by machine learning are UltraSound, magnetic 


Bulletin of Electr Eng & Inf, Vol. 11, No. 3, June 2022: 1358-1369 


Bulletin of Electr Eng & Inf ISSN: 2302-9285 O 1361 


resonance imaging scans, CT-scans and X-ray. Positron emission tomography (PET) scans, retinal 
photography, histology slides, and dermoscopy images [21]. In this work, X-ray and CT scan digital images 
have been applied in diagnosing chest diseases like COVID-19. DL models are widely used in processing 
medical images for applications like image segmentation, classification and prediction with a high accuracy. 
These models can automatically learn the features of an image like the lines, edges and merge these with 
other features like shapes found in the subsequent layers. In general, DL networks may recognize unique 
features that are hard to identify it using visual recognition [22], [23]. In this section, the proposed method 
that are used to detect COVID-19 cases is described. 


3.1. Proposed method 

In this paper, different DCNNs are used to detect COVID-19 patients using two learning techniques, 
supervised and unsupervised. The first one uses DCNNs for image classification using the fully connected 
networks. Meanwhile, the second one uses learned features from DCNNs to retrieve the most similar images. 
Figure 1 shows framework stages including image classification and image retrieval. The first stage is image 
pre-processing where the images are cropped, converted from gray to RGB, and resized to unify them as 
required by the DCNNs due to chest CT-scan images or X-rays have come with different dimensions and 
sizes. In our previous work, an encoder-decoder CNN called SegNet image segmentation as shown in 
Figure 2 is applied. It is based on the VGG-19 network which is used to crop the images, classify pixel-wise 
and yield some useful features to locate an accurate boundary as depicted in Figure 3(a) image preparation 
phase and Figure 3(b) crop the rib cage portion [8]. This process is important to increase the accuracy of the 
networks and reduce their training time. The second stage is feature extraction where main parts are detected 
in multiple hidden layers of each DCNNs. The third stage is the performance evaluation related to the 
classified and retrieved images where the label of an image is presented or not. 
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Figure 2. An encoder-decoder CNN called SegNet image segmentation block diagram [8] 
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(b) 


Figure 3. Block diagram of automatic X-ray segmentation (a) image preparation phase and (b) crop the rib 
cage portion 


4. EXPERIMANTAL DATASETS 

In this paper, two datasets have been used to diagnose patients infected with COVID-19 during the 
outbreak of this virus. The chest images dataset is taken from a CT-scan and X-ray which are illustrated in 
Table 1. For CT-scan chest images, the dataset has been confirmed by a senior radiologist in Tongji Hospital, 
Wuhan, China [24], [25]. for the period 19th of January to 15th of March. It is considered as one of the 
largest public COVID-19 CT-scan images collection. Figure 4 shows samples from CT-scan chest images 
dataset where 1 mean infected with COVID-19 and 2 for non COVID-19 [24], [25]. There are 349 chest 
images that are have COVID-19 (COVID-19*) and 397 with no COVID-19 chest images (COVID-19°) or 
that are either normal or contain other types of diseases. All of these images have different sizes. The 
CT-images have resized depending upon the model that has trained using the DCNNs models. For example, 
for the AlexNet network, the images are resized to 227x227 while they are 224x224 for ResNet networks. 
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Table 1. Datasets details 


Samples Radiograph image type Number Source 
COVID-19* CT-scan 349 [23], [24] 
COVID-19 397 

COVID-19* X-ray 167 [26], [27] 
COVID-19" 184 


Another dataset has been used in this work are the chest X-ray images. The dataset is collected from 
two online available datasets [25], [26]. For COVID-19 images, this dataset has two folders. The first folder 
contains COVID-19 chest images and the non COVID-19 images. The second folder also contains both 
COVID-19 and non COVID-19 images. Both folders contain 351 images that are divided into 184 images non 
COVID-19, and 167 images COVID-19. A Sample from this chest image dataset is shown in Figure 5 where 1 
means chest images for people infected with COVID-19 and 2 for non COVID-19 people. In the next section, 
the experimental results are conducted on these datasets to see the performance of the DCNNs models. 
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Figure 4. Samples from CT-scan chest images Figure 5. A sample from X-ray chest image dataset, 
dataset where 1 mean infected with COVID-19 and 2 where | mean infected with COVID-19 and 2 for 
for non COVID-19 [24], [25] non COVID-19 [26], [27] 


5. EXPERIMENTAL RESULTS 

In this work, exhausted experiments are conducted in terms of image classification and retrieval 
using chest images. In these experiments, DL models are used and implemented in a framwork of the 
proposed medical system. In addition, the performance evaluation of the image classification and retrieval are 
presented and discussed in this section. 


5.1. Chest image classification performance evaluation 

The confusion matrix is computed to find the accuracy for the classifier models. For example, a 
confusion matrix for 2-by-2 for any two classes is shown in Table 2 and can be enlarged into n-by-n classes. 
Four terms have been used for image classification. These terms are used to calculate the recall and precision 
to estimate the performance of the proposed models as (1), (2), and [28]. 


Recall = —— (1) 
TP+FN 
Precision = —— (2) 
TP+FP 


Where TP is true positive, TN is true negative, FN is false negative, FP and is false positive. 


Table 2. Confusion matrix 
Predicted class C1 C2 

Actual Cl TP FN 

class C2 FP TN 


5.2. Chest image retrieval performance evaluation 

Essentially, CBIR matches extracted features of a demanded image to those of database images 
using one of the similarity measures and returns the most similar images as a ranked list Figure 1. A common 
evaluation measure for image retrieval is clearified as (3), [28]. 
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RN 

P(C) =z (3) 
Where, P is the precision of image retrieval, RN is the number of relevant retrieved images that belongs to 
class C; TRN is the total number of images in the retrieved list as (4). 


Bea? 
AP(C) = 7# (4) 


Where, AP is the average precision of retrieved images, Pẹ is the precision of k image in the class C, and tc 
is the total class of images as (5). 


we, APg 
MAP op y = = 


(5) 
Where, MAP is the mean average precision of Top n retrieved image, n = 5,10,20,...,100), AP, is the 
average precision of q class image, tb=total classes in the database. 


5.3. Experimental setup and performance analysis 

In this section, the configurations of the experiments and the analysis of the results are described in 
details. The experiments have been performed to evaluate our proposed medical retrieval algorithm from 
various perspectives. For performance analysis, the algorithm is implemented using DL models toolbox in a 
Matlab software package and applied on chest radiograph images. The software is operated on a computer 
device with specification memory (16 GB), processor Intel ® Core ™ 17-8550U and CPU @ 1.80 GHz. In 
addition, proposed models were compared to some previous DCNNs. Data augmentation is used to increase 
the generalization of the models. At the end, a decision is made about the patient's case if he/she has got 
COVID-19 or not in terms of classification. Also, the most similar top 5 is retrieved images in terms of 
CBIR. 

Different CNNs models are investigated and compared with other previous works. These models are 
used to classify chest images and exploit them to learn the features of the images. In this work, AlexNet, 
GoogleNet, ResNet-50 CNNs models are applied and compared. These models are used in distinguishing 
patients with COVID-19 disease, healthy and chest disease patients. Data augmentation operation is also used 
at training time to make more data and to obtain a good accuracy. The final step is applied to classify the 
patients with COVID-19 or not using a fully connected layer. For the fully connected layer, the training 
parameters for the system are setup such as the mini-batch stochastic gradient descent algorithm is 
implImented, the learning rate is 0.00001, the batch size is 24, and the weights of the learning rate are set to 
10. In the following sections, we will introduce the experimental settings and the results for these approaches. 


5.4. Image classification 

In this section, the performance of DCNNs models to diagnose COVID-19 chest images has been 
examined. Many experiments are conducted on both CT-scan and X-ray chest image datasets. The datasets 
are divided into 70% for training, 30% for validation and testing data. In the validation phase, data is 
validated for model generalization or early stopping during the training process. On the other hand, in the 
testing phase, the data are tested for other than the training and validating process. There is no overlap 
between the training, validation, and testing dataset in building reliable and efficient results about the models. 
As stated earlier, metrics such as the confusion matrix is used for the performance evaluation of the models. 
For example, Table 3 shows the confusion matrix for the performance of all models on CT-scan chest 
images. As mentioned earlier, the most common DCNNs used in this study are AlexNet, GoogleNet, and 
ResNet-50. For the results evaluation performance, the accuracy of the testing data has been used. 


Table 3. Confusion matrices related to the COVID-19 class, for the evaluation performing for testing dataset 


Target class Target class Target class 

1 2 1 2 1 2 
Output 1 95 10 90.5% Output 1 76 29 72.4% Output 1 53 52 50.5% 
class 424% 45% 9.5% class 33.9% 12.9% 27.6% class 23.7% 23.2% 49.5% 
2 17 102 85.7% 2 12 107 89.9% 2 20 99 83.2% 
7.6% 45.5% 14.3% 5.4% 47.8% 10.1% 8.9% 44.2% 16.8% 
84.8% 91.1% 87.9% 86.4% 78.7% 81.7% 72.6% 65.6% 67.9% 
15.2% 8.9% 12.1% 13.6% 21.3% 18.3% 274% 344% 32.1% 

GoogleNet AlexNet ResNet 
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5.4.1. CT-scan images results 

For the CT-scan chest, images based on Tongji Hospital, Wuhan, China database is used. The 
results listed in Table 4 are for three models mentioned earlier: AlexNet, GoogleNet, and ResNet-50 models. 
The results are for the COVID-19 and non COVID-19 datasets. It is observed that obtained result from 
GoogleNet is more effective than other DL models in distinguishing COVID-19 and non COVID-19 
CT-scan chest images since it gives better generalization and avoids overfitting. Based on the confusion 
matrix, the accuracy, recall and precision were calculated. Table 5 illustrates the accuracy, recall and 
precision based on the confusion matrix of the proposed DL models. 


Table 4. DL models performance accuracy Table 5. The accuracy, recall and precision based on 
for image classification using Tongji confusion matrix of DL models on the CT scan chest images 
Hospital, Wuhan, China database [24] based on Tongji Hospital, Wuhan, China database [24] 
Method Accuracy Method Accuracy Recall Precision 
GoogleNet 87.95% GoogleNet 87.95% 0.8482, 0.9107 0.8810 
AlexNet 81.70% AlexNet 81.70% 0.8636, 0.7868 0.8115 
ResNet-50 67.86% ResNet-50 67.86% 0.7260, 0.6556 0.6683 


5.4.2. X-ray images results 

For the X-ray chest image datasets, the results are much better than with CT-scan images in 
distinguishing between the COVID and non COVID-19 infected patients. Table 6 shows the results of DL 
models on these images. As can be seen from the Figure, the accuracies are very high using these models on 
the X-ray chest images. The experiments shows that if the number of epochs is increased for all these models, 
the classification is almost perfect. 


Table 6. Performance accuracy of DL models as image classification based on X-ray chest images 
Method Accuracy 
GoogleNet 100% 
AlexNet 99% 
ResNet-50 97% 


Table 7 compares the performance accuracy of our models (AlexNet, GoogleNet and ResNet-50) 
with three models suggested in Zhao et al. [24] for the cases of infected COVID-19 or not infected. All of 
these results are based on Ct-scan images database on Tongji Hospital, Wuhan, China. Different models are 
addressed in Zhao et al. [24] such as ResNet-50, VGG-16 and CRNet. As can be seen from the table, our 
model based on GoogleNet successfully gave the best classification accuracy (about 87.95%). On the other 
hand, the same comparison is conducted on the X-ray images dataset. The results are much better than 
CT-scan images dataset for all models. As mentioned in section 4, the X-ray images dataset is collected from 
two online available datasets [26], [27]. Table 8 shows the performance accuracy of our proposed DCNNs 
with other models such as that is suggested [29]. 


Table 7. Performance comparisons accuracy of DL networks for image classification using Tongji Hospital, 
Wuhan, China database [24], [25] 


Method Accuracy 
ResNet-50 [24] 66% 
VGG-16 [24] 69% 
CRNet [24] 72% 
AlexNet 81.70% 
GoogleNet 87.95% 
ResNet-50 78.13% 


Table 8. Performance comparisons accuracy of DL models as image classification based on X-ray chest 
images [26], [27] 


Method Accuracy 
GoogleNet 100% 
AlexNet 99% 
ResNet-50 97% 
DAG3Net [30] 96.58 
ResNet 101 [30] 100% 


Inception V3 [30] 99.72% 
GoogleNet [30] 98.08% 
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The most interesting finding from this study, the difference in the performance of the models for 
both kinds of medical images (for COVID-19 and non COVID-19 infected patients) is that the experiments 
of training X-ray medical images are much faster than CT-scan images since the time and number of epochs 
taken on the proposed models is far less for X-ray chest images. For instance, it takes less than a minute in 
AlexNet. Also, the accuracies using X-ray chest images are much better than on the CT-scan chest images as 
shown in Tables 4 and 6. However, these days, CT-scan images are the best method to diagnose COVID-19 
on infected patients because of their high sensitivity to X-ray images. Besides, X-ray chest images are only 
used by radiologists to find features that could be significant in screening COVID-19. Though the features 
may not be that accurate as they can be typical pneumonia and other pulmonary manifestations [30]. It can be 
concluded that the results indicate high accuracy can be found using DCNNs. The results also indicate that 
these networks are superior in diagnosing medical images classification is almost successful. 


5.5. Image retrieval 

As stated earlier, Figure 1 shows the steps of proposed retrieval algorithm. At Off-Line stage, 
images from two mentioned datasets are processed by pre-processing, different DCNNs exploited to learn 
features. At On-Line stage, a query image is processed following the same steps to learn features. A distance 
function is used to find the similarity between the query image feature and database image features. Then, a 
ranked list of the most top 5 similar images is returned that may help a radiologist or specialist doctor to 
make the right decision. Since the similarity measurement is another issue in CBIR [31], [32] our 
experiments tested City-block (DF 1) and cosine (DF2) distance functions as (6) and (7). 


DF, = yee [sins — XpBim; (6) 


T 
DF, a 1 = XQIm*DBIm (7) 


[Comtim d@DB1m*b Bim) 
Where Xgim and Xpgim refer to query image and database image feature vectors with m dimension. 


5.5.1. CT-scan images results 

For the CT-scan chest images based on Tongji Hospital, Wuhan, China database, Table 9 shows 
mean average precisions (MAP) using AlexNet, GoogleNet and ResNet-50 models. The results are for the 
COVID-19 (+) and COVID-19 (-) image classes. It is clear that GoogleNet satisfied highest performance 
using City-block distance function, although GoogleNet feature has 2D compared to ResNet-50 and AlexNet 
features which have 2049D and 4096D respectively. This means that GoogleNet feature need less time than 
two other and it is better to use. Moreover, we can judge the significant difference between GoogleNet and 
other two models by using a f-test statistical method [33] as (8). 


Mı-M2 
(K1-1)S?+(K2-1)s2 ( 144 ) 
K1+K2-2 Kı K2 


Where M, and M, are the sample precision rates (P), Sı and S2 are standards deviations, and K, and K, are 
the sample sizes. The t-test regards two hypotheses, the null hypothesis (Ho) where M, — M, = 0 and 
alternative hypothesis (Ha) where M, — M, + 0. P-value of the test is the probability of observing a test. 
Small values of p refer to that the null hypothesis is rejected at significance level 0.05. 

Tables 10 and 11 show results of the t-test for GoogleNet and ReseNet precision samples and 
GoogleNet and AlexNet precision samples respectively. Both tests indicate that the difference between the 
performance of GoogleNet model and others is significant at P-value less than (0.00001). Thus, we can 
deduce the model produced a feature with low dimension (2) but it can achieve effective image retrieval. This 
means the inception block in the model exploiting operations like split, merge and transform. 


t= 


(8) 


Table 9. MAPs using GoogleNet, ResNet-50, AlexNet models with DF; and DF2 for CT-scan dataset 


Method The case of disease Top 5 
GoogleNetDF; COVID-19* 84% 
COVID-19 81% 

ResNetDF> COVID-19* 70% 
COVID-19° 62% 

AlexNetDF, COVID-19* 61% 
COVID-19" 64% 
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Table 10. The ¢-test result using Google and ResNet- Table 11. The ¢-test result using Google and AlexNet 


50 sample precision rates sample precision rates 
Value COVID-19 No COVID-19 Value COVID-19 No COVID-19 
t-value 5.08518 6.20932 t-value 9.64735 6.48786 
P-value <0.00001 <0.00001 P-value <0.00001 <0.00001 


5.5.2. X-ray images results 

For the X-ray chest image datasets, Table 12 illustrates MAPs for Top5 retrieved images by 
applying AlexNet, ResNet-50 and GoogleNet models. The outcomes are presented for COVID-19* and 
COVID-19 image classes. At the first view, we can see the performance of all models are more than 94%. 
More further, MAPs of Resenet-50 and AlexNet are approximately equal using City-block and Cosine 
distance functions respectively. However, the dimension of a feature that is produced by AlexNet higher than 
that is produced by Resnet-50. On the other hand, the accuracy of GoogleNet is less about 3% comparing 
with Resnet-50 and AlexNet, but the dimension of obtained feature is only 2D. Here, we also conclude that 
GoogleNet can save time and achieve promised performance. 

Extra experiments conducted on above extracted features using voting in k-nearest neighbors 
(k-NN). A high improvement observed for CT-scan images as illustrated in Table 13 due to k-NN is 
supervision learning technique which means the label of class is present. Consequently, the accuracy of 
classifier supported to achieve more accuracy. In Table 14 of confusion matrix, we can see about 50 images 
from COVID and non COVID Classes conflict with the verse’s classes using GoogleNet. Meanwhile, the 
number of confusion images increased about 20 to 70 using ResNet and AlexNet because the GoogleNet 
model produces a feature that less in length than those of two others and more discriminant. 

On the other hands, Table 15 illustrates the performance of k-NN classifier for the three models 
where classification rates of ResNet-50 and AlexNet satisfied about 1 and 3 per cent more for COVID 
images comparing with retrieval rates Table 11. Meanwhile, there is no change with the GoogleNet model. 
However, still this model has the advantages in terms of distinguishable and efficient feature (2D) to achieve 
95-96% for COVID and non COVID images. Hence, Table 16 shows confusion between two classes where 8 
COVID images are classified as non COVID and 7 non COVID images are classified as COVID. 


Table 12. MAPs using GoogleNet, Table 13. Recall (A-NN) using GoogleNet, 
Resnet, AlexNet models with DF; and DF: for ResNet-50, AlexNet models with DF, and DF» for 
X-ray dataset CT-scan dataset 
Method The case of disease Top 5 Method The case of disease 5-KK 

ResNetDF; COVID-19* 0.98 GoogleNetDF; COVID-19* 0.87 
COVID-19° 0.99 COVID-19° 0.85 

AlexNetDF, COVID-19* 0.97 ResnetDF; COVID-19* 0.82 
COVID-19° 0.99 COVID-19° 0.74 

GoogleNetDF, COVID-19* 0.95 AlexNet DF; COVID-19* 0.69 
COVID-19" 0.96 COVID-19 0.70 


Table 14. Confusion matrix 


Class 1 2 Class 1 2 Class 1 2 
1 345 52 1 325 72 1 212. 125 
2 53 296 2 89 260 2 103 246 
GoogleNet ResNet-50 AlexNet 


Table 15. Recall (k-NN) using GoogleNet, ResNet-50, AlexNet models with DF; and DF: for X-ray dataset 


Method The case of disease 5-KK 
ResNetDF, COVID-19* 0.99 
COVID-19- 0.99 

AlexNetDF, COVID-19* 0.99 
COVID-19" 0.99 

GoogleNetDF, COVID-19* 0.95 
COVID-19° 0.96 


Table 16. Confusion matrix 


Class 1 2 Class 1 2 Class 1 2 
1 159 8 1 165 2 1 165 2 
2 7 177 2 2 182 2 2 182 
GoogleNet ResNet-50 AlexNet 
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6. CONCLUSION AND FUTURE WORKS 

In this paper, a retrieval algorithm was developed that may help a radiologist or specialist doctor to 
make the right decision. The proposed model was applied to both the chest radiographs X-ray, and CT-scan 
to diagnose the virus, based on different DCNNs (Resnet-50, AlexNet, and GoogleNet) models. Learned 
features that obtained from the models and similarity measures performed differently on X-ray and CT-scan 
datasets. However, GoogleNet produced less feature in length (2D) and achieved highest accuracy using 
CT-scan images. Meanwhile, it is just 3% less than other models using X-ray images. A system can be 
developed to make a progress of people's health when there is limited access to expert physicians. For future 
works, the proposed model can be used with large datasets and even include all chest diseases. 


ACKNOWLEDGEMENTS 
The authors of this work would like to thank Mustansiriyah University 
http://www.uomustansiriyah.edu.iq in Baghdad, Iraq for its support. 


REFERENCES 

[1] A. Shoeibi et al., “Automated detection and forecasting of Covid-19 using deep learning techniques: A review,” ArXiv preprint 
arXiv:2007.10785, p. 20, 2020, doi: 10.48550/arXiv.2007.10785. 

[2] L. O. Hall, R. Paul, D. B. Goldgof, and G. M. Goldgof, “Finding Covid-19 from chest x-rays using deep learning on a small 
dataset,” arXiv preprint arXiv:2004.02060, pp. 1-8, 2020, doi: 10.48550/arXiv.2004.02060. 

[3] J. D. B. Castro, R. Rei, J. E. Ruiz, S. A. Diaz, Pedro Achanccaray Canchumuni, L. F. Villalobos, Cristian Munoz Coelho, Felipe 
Borges Mendoza, and M. A. C. Pacheco, “A free web service for fast COVID-19 classification of chest X-ray images,” arXiv 
preprint arXiv:2009.01657, p. 14, 2020, doi: 10.48550/arXiv.2009.01657. 

[4] N. M. Elshennawy and D. M. Ibrahim, “Deep-pneumonia framework using deep learning models based on chest X-ray images,” 
Diagnostics, vol. 10, no. 9, p. 649, Aug. 2020, doi: 10.3390/diagnostics 10090649. 

[5] I. Chamveha and W. Tongdee, Trongtum Saiviroonporn, Pairash Chaisangmongkon, “Local adaptation improves accuracy of 
deep learning model for automated x-ray thoracic disease detection: a Thai study,” arXiv preprint arXiv:2004.10975, p. 9, 2020, 
doi: 10.48550/arXiv.2004.10975. 

[6] G. A. Shadeed, M. A. Tawfeeq, and S. M. Mahmoud, “Deep learning model for thorax diseases detection,” TELKOMNIKA 
(Telecommunication Computing Electronics and Control), vol. 18, no. 1, pp. 441-449, Feb. 2020, doi: 
10.12928/telkomnika.v18i1.12997. 

[7] G. A. Shadeed, M. A. Tawfeeq, and S. M. Mahmoud, “Automatic Medical Images Segmentation Based on Deep Learning 
Networks,” JOP Conference Series: Materials Science and Engineering, vol. 870, no. 1, p. 012117, Jun. 2020, doi: 10.1088/1757- 
899X/870/1/012117. 

[8] G. A. Shadeed and S. M. Tawfeeq, Mohammed A Mahmoud, “Diagnosing thorax diseases using deep learning models,” Journal 
of Engineering and Sustainable Development (JEASD), vol. 24, no. Special_ Issue_2020, pp. 1-5, 2020. 

[9] Y. Zhong, “Using deep convolutional neural networks to diagnose COVID-19 from chest X-ray images,” arXiv preprint 
arXiv:2007.09695, p. 14, 2020, doi: 10.48550/arXiv.2007.09695. 

[10] S. Albahli, “Efficient GAN-based chest radiographs (CXR) augmentation to diagnose coronavirus disease pneumonia,” 

International Journal of Medical Sciences, vol. 17, no. 10, pp. 1439-1448, 2020, doi: 10.7150/ijms.46684. 

[11] H. Ko et al., “COVID-19 pneumonia diagnosis using a simple 2D deep learning framework with a single chest CT image: model 

development and validation,” Journal of Medical Internet Research, vol. 22, no. 6, p. €19569, Jun. 2020, doi: 10.2196/19569. 

[12] D. Dansana et al., “Early diagnosis of COVID-19-affected patients based on X-ray and computed tomography images using deep 

learning algorithm,” Soft Computing, pp. 1-9, Aug. 2020, doi: 10.1007/s00500-020-05275-y. 

A. Jaiswal, N. Gianchandani, D. Singh, V. Kumar, and M. Kaur, “Classification of the COVID-19 infected patients using 

DenseNet201 based deep transfer learning,” Journal of Biomolecular Structure and Dynamics, pp. 1-8, Oct. 2020, doi: 

10.1080/07391102.2020.1788642. 

[14] V. Bhandi and S. D. K. A, “Covid-19 x-ray image retrieval using deep convolutional neural networks,” American Journal of 

Engineering Research (AJER), vol. 9, no. 7, pp. 47-55, 2020. 

F. Z. Benyelles, A. Sekkal, and N. Settouti, “Content Based COVID-19 Chest X-Ray and CT images retrieval framework using 

Stacked Auto-Encoders,” in 2020 2nd International Workshop on Human-Centric Smart Environments for Health and Well-being 

(IHSH), Feb. 2021, pp. 119-124, doi: 10.1109/THSH5 1661.2021.9378730. 

[16] A. Zhong et al., “Deep metric learning-based image retrieval system for chest radiograph and its clinical applications in COVID- 

19,” Medical Image Analysis, vol. 70, no. 1, p. 101993, May 2021, doi: 10.1016/j.media.2021.101993. 

[17] H. J. Hwang et al., “Content-based image retrieval of chest CT with convolutional neural network for diffuse interstitial lung 

disease: performance assessment in three major idiopathic interstitial pneumonias,” Korean Journal of Radiology, vol. 22, no. 2, 

pp. 281-290, 2021, doi: 10.3348/kjr.2020.0603. 

[18] O. F. Layode and M. Rahman, “A chest X-ray image retrieval system for COVID-19 detection using deep transfer learning and 

denoising auto encoder,” in 2020 International Conference on Computational Science and Computational Intelligence (CSCI), 

Dec. 2020, pp. 1635-1640, doi: 10.1109/CSCI51800.2020.00301. 

[19] M.R. Kapadia and C. N. Paunwala, “Content based medical image retrieval system for accurate disease diagnoses using modified 

multi feature fused Xception model,” / Indian Journal of Computer Science and Engineering (IJCSE), vol. 12, no. 1, pp. 89-100, 

2021, doi: 10.21817/indjcse/2021/v1211/21120119. 

T. Rajasenbagam, S. Jeyanthi, and J. A. Pandian, “Detection of pneumonia infection in lungs from chest X-ray images using deep 

convolutional neural network and content-based image retrieval techniques,” Journal of Ambient Intelligence and Humanized 

Computing, pp. 1-8, Mar. 2021, doi: 10.1007/s12652-021-03075-2. 

[21] J. Ker, L. Wang, J. Rao, and T. Lim, “Deep learning applications in medical image analysis,” IEEE Access, vol. 6, pp. 9375-9389, 
2017, doi: 10.1109/ACCESS.2017.2788044. 


Bulletin of Electr Eng & Inf, Vol. 11, No. 3, June 2022: 1358-1369 


Bulletin of Electr Eng & Inf ISSN: 2302-9285 O 1369 


[22] 
[23] 
[24] 
[25] 
[26] 
[27] 


[28] 


[29] 


[30] 


[31] 


[32] 


[33] 


S. Wang et al., “A deep learning algorithm using CT images to screen for Corona virus disease (COVID-19),” European 
Radiology, pp. 1-9, Aug. 2021, doi: 10.1007/s00330-021-07715-1. 

M. Loey, F. Smarandache, and N. E. M. Khalifa, “Within the lack of chest COVID-19 X-ray dataset: a novel detection model 
based on GAN and deep transfer learning,” Symmetry, vol. 12, no. 4, p. 651, Apr. 2020, doi: 10.3390/sym12040651. 

J. Zhao, Y. Zhang, X. He, and P. Xie, “COVID-CT-dataset: A CT scan dataset about COVID-19,” 2020. [Online]. Available: 
https://github.com/UCSD-AI4H/COVID-CT. 

X. He et al., “Sample-efficient deep learning for COVID-19 diagnosis based on CT scans,” Medrxiv, p. 10, 2020, doi: 
10.1101/2020.04.13.20063941. 

J. P. Cohen, P. Morrison, L. Dao, K. Roth, K. Roth, and M. Ghassemi, “Covid-19 image data collection: prospective predictions 
are the future,” arXiv preprint arXiv: 2006.11988, pp. 1-38, 2020, doi: 10.48550/arXiv.2006.11988. 

P. Mooney, “Chest X-ray pneumonia,” 2020. [Online]. Available: https://www.kaggle.com/paultimothymooney/chest-xray- 
pneumonia). 

H. Müller, W. Müller, D. M. Squire, S. Marchand-Maillet, and T. Pun, “Performance evaluation in content-based image retrieval: 
overview and proposals,” Pattern Recognition Letters, vol. 22, no. 5, pp. 593-601, Apr. 2001, doi: 10.1016/S0167- 
8655(00)001 18-5. 

G. A. Shadeed, A. A. Jabber, and N. H. Alwash, “I. Covid-19 detection using deep learning models,” in 2027 Ist Babylon 
International Conference on Information Technology and Science (BICITS), Apr. 2021, pp. 194-198, doi: 
10.1109/BICITSS1482.2021.9509874. 

J. E. Lujan-Garcia, M. A. Moreno-Ibarra, Y. Villuendas-Rey, and C. Yafiez-Marquez, “Fast COVID-19 and pneumonia 
classification using chest X-ray images,” Mathematics, vol. 8, no. 9, p. 1423, Aug. 2020, doi: 10.3390/math8091423. 

H. Du, H. Al-Jubouri, and H. Sellahewa, “Effectiveness of image features and similarity measures in cluster-based approaches for 
content-based image retrieval,” in Mobile Multimedia/Image Processing, Security, and Applications, International Society for 
Optics and Photonics, May 2014, vol. 9120, p. 912008, doi: 10.1117/12.2057721. 

H. A. Al-Jubouri and S. M. Mahmmod, “A comparative analysis of automatic deep neural networks for image retrieval,” 
TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 19, no. 3, pp. 858-871, Jun. 2021, doi: 
10.12928/telkomnika.v19i3.18157. 

A. M. Graziano and M. L. Raulin, Research methods: a process of inquiry. Prentice Hall PTR, 2012, 2012. 


BIOGRAPHIES OF AUTHORS 


Sawsan M. Mahmoud Ô fJ BB P received her B.Sc. in Computer Science from University 
of Technology/Baghdad, Iraq in 1994. She obtained her M.Sc. from University of Baghdad n 
1998. Her Ph.D. degree in Computational Intelligence is obtained from Nottingham Trent 
University, Nottingham, UK in 2012. Sawsan joined the academic staff of Mustansiriyah 
University/Engineering College in 1994. Her research interests include computational 
intelligence, ambient intelligence (smart home and intelligent environment), wireless sensor 
network, data mining, and health monitoring. She can be contacted at email: 
sawsanmousal 1 @gmail.com. 


Hanan A. S. Al-Jubouri © EJ BB P received her B.Sc. in Computer Science from 
University of Technology/Baghdad, Iraq in 1994. She obtained her M.Sc. from University of 
Technology in 2001. Her Ph.D. degree in Information Systems, Buckingham University, 
Buckingham, UK, 2015. Hanan joined Mustansiriyah University/Engineering College in 
1994 as a member of the academic staff. Her research interests are mainly content-based 
image retrieval and data mining. She can be contacted at email: hananaljubouri@gmail.com. 


Tawfeeq E. Abdoulabbas © EVES P received his B.Sc in Electrical Engineering from 
Mustansiriyah University/Baghdad, Iraq in 1992. In 1996, he obtained his M.Sc. from 
University of Technology/Baghdad, Iraq in Electrical and Engineering science. Tawfeeq 
joined the academic staff of Mustansiriyah University/College of Engineering in 2012. His 
research interests are: power management system, smart home, smart grids. He can be 
contacted at email: tawfikenad@gmail.com. 


Chest radiographs images retrieval using deep learning networks (Sawsan M. Mahmoud) 


